<html>
<head><meta charset="utf-8"><title>mutex impls in std · t-libs · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/index.html">t-libs</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/mutex.20impls.20in.20std.html">mutex impls in std</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="211817652"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/mutex%20impls%20in%20std/near/211817652" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Sam Rijs <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/mutex.20impls.20in.20std.html#211817652">(Sep 30 2020 at 20:43)</a>:</h4>
<p>Hi team!</p>
<p>I've recently started to run into a couple of issues with std locks, mostly the non-<code>const</code>-iness of <code>::new</code> as well as the generally slow performance on macOS. Motivated by this, I read up on the adoption of parking_lot into  std and was sad to see that it looks like it might've stalled, perhaps partly due to time/priority constraints and partly due to the complexity of the implementation?</p>
<p>I was wondering if/how there's ways I could help resolve this. Some ideas I have (perhaps you have other/better ones?):</p>
<ol>
<li>Change the <code>Box</code> alloc in <code>::new</code> to be lazy, so that it can be <code>const</code>? (feels meh, also probably somewhat slower?)</li>
<li>Contribute a "simpler" lock impl that could be more easily audited/moved into <code>std</code>?</li>
<li>Help make <code>parking_lot</code> more suitable towards a <code>std</code> inclusion (but don't know how?)</li>
</ol>
<p>Especially on (2) I have some ideas based on per-lock linked-lists rather than a central hash table that I think could be a lot simpler and offer similar performance to <code>parking_lot</code> at the expense of a few more bytes (1 word for mutex and 2 for rwlock/condvar).</p>
<p>Do you have any input on this? Any feeling on which approach might be the most fruitful? Is this something worth pursuing via RFC?</p>



<a name="211826324"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/mutex%20impls%20in%20std/near/211826324" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Steven Fackler <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/mutex.20impls.20in.20std.html#211826324">(Sep 30 2020 at 20:58)</a>:</h4>
<p>I think working on parking_lot's integration would be the best path forward</p>



<a name="211826640"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/mutex%20impls%20in%20std/near/211826640" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Sam Rijs <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/mutex.20impls.20in.20std.html#211826640">(Sep 30 2020 at 21:01)</a>:</h4>
<p>Thanks! Do you have any pointers as to what would need to be done? From what I could glean from the existing issues/PR it seemed part of the issue was reviewing the code? I‘m also a bit concerned that parking lot seems to have gained a lot of features that std might not use, but perhaps that‘s overthinking it?</p>



<a name="211828238"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/mutex%20impls%20in%20std/near/211828238" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Steven Fackler <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/mutex.20impls.20in.20std.html#211828238">(Sep 30 2020 at 21:16)</a>:</h4>
<p>I was not following the integration super closely, but <span class="user-mention" data-user-id="143274">@Amanieu</span> would probably be the right person to ask.</p>



<a name="211839858"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/mutex%20impls%20in%20std/near/211839858" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Amanieu <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/mutex.20impls.20in.20std.html#211839858">(Sep 30 2020 at 22:06)</a>:</h4>
<p>It's been a while since I last looked into this. I would suggest starting by reading Alex's review of parking_lot here: <a href="https://github.com/faern/parking_lot/pull/1">https://github.com/faern/parking_lot/pull/1</a></p>



<a name="211839979"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/mutex%20impls%20in%20std/near/211839979" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Amanieu <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/mutex.20impls.20in.20std.html#211839979">(Sep 30 2020 at 22:07)</a>:</h4>
<p>One of the concerns was the use of the global allocator for the global hash table but I think we can probably replace the global hash table with a fixed-size table. The table uses chaining so it isn't really capacity-bound.</p>



<a name="211840131"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/mutex%20impls%20in%20std/near/211840131" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Amanieu <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/mutex.20impls.20in.20std.html#211840131">(Sep 30 2020 at 22:09)</a>:</h4>
<p>cc <span class="user-mention" data-user-id="117909">@faern</span> who was originally working on this.</p>



<a name="211840402"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/mutex%20impls%20in%20std/near/211840402" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Sam Rijs <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/mutex.20impls.20in.20std.html#211840402">(Sep 30 2020 at 22:13)</a>:</h4>
<p>Interesting point about the allocator, but why wouldn‘t the current use of box inside std::sync::Mutex run into the same problem?</p>
<p>I‘ll be sure to check out the review you linked, thanks!</p>



<a name="211848696"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/mutex%20impls%20in%20std/near/211848696" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Sam Rijs <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/mutex.20impls.20in.20std.html#211848696">(Sep 30 2020 at 22:34)</a>:</h4>
<p>Also, if we use chaining, don‘t we need to use something like thread_local! for the chain nodes, which iirc also allocates? Or are both of these using a different allocator than the one which can be overwritten?</p>



<a name="211848940"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/mutex%20impls%20in%20std/near/211848940" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Sam Rijs <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/mutex.20impls.20in.20std.html#211848940">(Sep 30 2020 at 22:38)</a>:</h4>
<p>(Hmm, I suppose you could stack-allocate the nodes if you‘re careful?)</p>



<a name="211903267"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/mutex%20impls%20in%20std/near/211903267" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Amanieu <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/mutex.20impls.20in.20std.html#211903267">(Oct 01 2020 at 12:44)</a>:</h4>
<p>You can definitely stack allocate once you remove resizing from the global hashtable. At the moment the <code>thread_local</code> is used to track the max number of live threads which is used to resize the hash table as needed.</p>



<a name="212063636"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/mutex%20impls%20in%20std/near/212063636" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/mutex.20impls.20in.20std.html#212063636">(Oct 02 2020 at 11:49)</a>:</h4>
<p><span class="user-mention" data-user-id="348117">@Sam Rijs</span> I've been working slowly in this direction the past few weeks:</p>
<ul>
<li><a href="https://github.com/rust-lang/rust/pull/76932">https://github.com/rust-lang/rust/pull/76932</a></li>
<li><a href="https://github.com/rust-lang/rust/pull/77147">https://github.com/rust-lang/rust/pull/77147</a></li>
<li><a href="https://github.com/rust-lang/rust/pull/76919">https://github.com/rust-lang/rust/pull/76919</a></li>
<li><a href="https://github.com/rust-lang/rust/pull/77380">https://github.com/rust-lang/rust/pull/77380</a></li>
</ul>



<a name="212063828"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/mutex%20impls%20in%20std/near/212063828" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/mutex.20impls.20in.20std.html#212063828">(Oct 02 2020 at 11:51)</a>:</h4>
<p>The path from this towards const Mutex::new is not clear yet, but these steps should help regardless. That path could be parking_lot integration, or could be a simple futex based implementation for Linux, and some yet-to-be-determined solution for other unixes. At least on Windows (and wasm and cloudabi) Mutex::new can be const after these changes.</p>



<a name="212064201"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/219381-t-libs/topic/mutex%20impls%20in%20std/near/212064201" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/219381-t-libs/topic/mutex.20impls.20in.20std.html#212064201">(Oct 02 2020 at 11:56)</a>:</h4>
<p>Another interesting thing: Since <a href="https://github.com/rust-lang/rust/pull/76919">https://github.com/rust-lang/rust/pull/76919</a>, std's version of parking/unparking no longer depends on std::sync's mutexes and condvars on Linux. I will continue that work to make that the case on all platforms. Once that's the case, Mutex <em>could</em> be modified to contain a std::lazy::SyncOnceCell. (Right now that's impossible, because SyncOnceCell uses a Once, which uses parking/unparking, which uses a Mutex on some platforms.) Not sure if that's a good idea, but it would make a lazy allocating Mutex implementation very easy.</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>